A Hierarchical Parsing Approach with Punctuation Processing for Long Chinese Sentences

(National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100080, China) Abstract: Based on the analysis of the usage and the syntactic function of Chinese punctuations, this paper proposes a new hierarchical approach to parsing the long Chinese sentences. In traditional parsing approaches, the parsing procedure is performed on one-level and the punctuation marks are not specially treated. Correspondingly, in our approach the complex long Chinese sentences are broken into sub-sentences or units (say ‘units’ hereafter) by using of the punctuation marks with special functions, so that the original whole sentence is parsed unit by unit. This idea of ‘dividing-and-ruling’ greatly reduces the difficulty in the traditional parsing approaches to recognize the syntactic relationship between the sub-sentences and phrases or inside the sub-sentences or phrases. And also, in our approach the grammatical rules with punctuation marks and their probabilities are extracted from the large scale Treebank, which are very beneficial for the syntactic disambiguation. Our experimental results have shown that comparing with the traditional Chart parsing algorithm, our approach can significantly reduce the time consumption and the numbers of ambiguous edges, and about 7% of the correct rate and the recall rate have been increased in parsing the long Chinese sentences .